OPPENHEIM CONJECTURE FOR PAIRS CONSISTING OF A LINEAR 

FORM AND A QUADRATIC FORM 



ALEXANDER GORODNIK 

Abstract. Let Q be a nondegenerate quadratic form, and X is a nonzero linear form of 
dimension d > 3. As a generalization of the Oppenheim conjecture, we prove that the set 
{{Q{x), L{x)) : X e Z''} is dense in provided that Q and L satisfy some natural conditions. 
The proof uses dynamics on homogeneous spaces of Lie groups. 



1. Introduction 

It was proved by Margulis |Ma89j (see |Ma97j for an up-to-date survey) that if Q is a real 
indefinite nondegenerate quadratic form of dimension li > 3 which is not proportional to a 
rational form, then {Q{x) : x G Z'^} is dense in M. A similar problem was considered for 
pairs {Q, L) where Q is a quadratic form as above, and L is a nonzero linear form. The 
known results are limited to dimension 3. It was proved by Dani and Margulis [D M90j 
that {{Q{x),L{x)) : X G Z^} is dense in jf 

no nonzero linear combination of Q and L is 
rational, and the plane {L = 0} is tangent to the surface {Q = 0}. Clearly, the first condition 
is neccesary for the density to hold. The second condition guarantees that the stabilizer in 
SL(3,]R) of the pair {Q,L) is unipotent, so that the results on orbits of unipotent flows can 
be used. One can hope to remove the second condition. However, Dani proved in IDaOOj 
that if the surface {Q = 0} and the plane {L = 0} intersect transversally, the density can fail 
for a set pairs of full Hausdorff dimension. On the other hand, it is easy to see using Moore 
ergodicity criterion that the density holds for a set of pairs of full measure provided that the 
surfaces {Q = 0} and {L = 0} have nonzero intersection. 

Denote by ■p(Z°') the set of primitive integer vectors in Z"'. The results mentioned above 
still hold when Z"' is replaced by V{7/'). 

In this paper we prove a density result for pairs consisting of a linear form and a quadratic 
form of dimension c? > 4: 

Theorem 1. Let Q he a nondegenerate quadratic form, and L he a nonzero linear form in 
dimension d> A. Suppose that 

1- Q\l=o is indefinite. 

2. For every {a, (3) ^ (0,0), with a,/? G M, aQ + {31? is not rational. 
Then {{Q{x),L{x)) : x G P(Z^}) is dense m M? . 

For (i = 3, condition (1) implies that the surfaces {(5 = 0} and {L = 0} intersect transver- 
sally. Therefore, it follows from the result of Dani |DaOOj mentioned above that the analogue 
of Theorem n does not hold for = 3. 
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It is easy to see that the second condition is neccesary for the conclusion of the theorem 
to hold. A condition similar to the first condition is required to insure that 

{{Q{x),L{x)) : X G M'^} =Ml 

It is possible that the first condition can be weakened (see Conjecture [121 below). 

Acknowledgment: The author is deeply grateful to his advisor V. Bergelson for sug- 
gesting the problem as well as for many helpful discussions. 

2. Canonical forms 

Let Qi, i = 1,2, be quadratic forms, and Lj, i = 1,2, be linear forms of dimension d. We say 
that the pairs {Qi,Li) and {Q2,L2) are equivalent if Qi{x) = \Q2{g-x) and Li{x) = jiL2{g-x) 
for some A, /i G M — {0} and g G SL(r;,, M). That is, they can be transformed into each other 
by a linear change of coordinates and scaling. 

Proposition 2. Every pair {Q,L), where Q is a nondegenerate quadratic form, and L is a 
nonzero linear form, is equivalent to one and only one of the following pairs: 

(I) (x? + . . . + x2 - -...-xj, Xd) 

where s = 1, . . . ,d. 

(II) (x? + . . . + - x^+i - ... - x^_2 + ^d-iXd, Xd) where s = 0, . . . , [^] . 

Proof. Without loss of generality, L{x) = Xd- Applying a linear transformation in xi, . . . , Xd-i, 
we transform Q to the form 




Xd for some k < d — 1. 



Using linear transformations Xj — > Xj ± ^^d for i < k, we can make = for i < k. If Oj = 
for all i ^ d, then k = d — 1 because Q is nondegenerate. Thus, we are in the case (I). Let 
a; 7^ for some / < d. Making a linear change of variables x; Ylj (^jXj, we get 

Q = ^ ±X,^ + XiXd- 
i<k 

Since Q is nondegenerate, k = d — 2 and I = d — 1. This is the case (II). Note that a pair 
{Q, L) is of type (I) iff Q + aL"^ is degenerate for some a G M. In particular, this shows 
that pairs of type (I) and (II) are not equivalent. Signatures of quadratic forms Q and Q\l=q 
are invariants of equivalence with one possible exception that pairs with signatures (s, d — s) 
and [d — s, s) could be equivalent. For example, pairs of type (I) with s = and s = d 
are equivalent. Let {Q,L) be as in (I) with s = i ioi i ^ Q,d. Then Q\l=q has signature 
{i,d — i — 1). On the other hand, when s = d — i, Q\l=o has signature {d — i,i — 1). Clearly, 
these two cases are not equivalent unless n = 2i, and they coincide. When {Q, L) is of type 
(II), it is easy to see that the cases s = i and s = d — 2 — i are equivalent for i = 0, . . . , — 2. 
This finishes the proof. □ 

Using the explicit formulas for canonical forms, it is easy to prove the following lemma: 

Lemma 3. Let Q he a nondegenerate quadratic form, and L is a nonzero linear form such 
that Q\l=q is indefinite. Then 

{(g(x),L(x)) : X G M'^} =Ml 
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3. Reduction to lower dimension 

In this section, we prove Proposition |5 which reduces the proof of Theorem ^ to the case 
of dimension 4. 

Proposition 4. Let {Q, L) satisfy the conditions of Theorem // the dimension of the 
space is greater than 4, then there exists a rational subspace U of codimension 1 such that 
{Q\u,L\u) satisfies the same conditions. Moreover, U can he chosen such that {Q\u,L\u) is 
of type (I). 

Consider the set VL = {(£i,£2,^3)} C M.^"^ of triples of hnear forms of dimension d defined 
by the following conditions: 

1. £1,^2,^3 are linearly independent. 

2. L 7^ on the space {ii = : i = 1,2, 3}. 

3. Q is nondegenerate on the space {£ = : £ G T} for any subset T C {£1, £2, ^s}- 

4. {Q\e^=o,L\e^=o) is of type (I) for i = 1,2,3. 

5. (5|{£^=o}n{L=o} is indefinite for i = 1,2,3. 

Lemma 5. The set Q is nonempty and open. 

Proof. Let Qq be the set of triples of linear forms that satisfy conditions (1) and (2), and 
Qj the set of triples in Qq that satisfy condition (j), j = 3,4,5. Note that the set Qq is the 
complement of a proper algebraic set (i.e. a set defined by polynomial equations). 

The sets flj, j = 3,4,5, are not empty. This is easy to see using the canonical forms 
from Proposition 121 For example, when {Q,L) is the canonical form (II), (£i,£2;^3) ^ ^4 for 
ii{x) = Xd-i-i + Xd-i + Xd, i = 1, 2, 3. It is also clear that the sets flj, j = 3, 4, 5, are open. 
To show this for ^4, we use that a pair {Q, L) is of type (I) iff 

det(Q) ^ 0, det(QU=o) 7^ 0, L ^ 0. 

It suffices to show that VL^ and VL^ are dense. Indeed, then it follows that fl Vt^ is an open 
dense set, and ^2 = fl n is nonempty and open. 

We claim that for every T C {ii,i2,^3}, the set of triples (^1,^25 ^3) G ^0 such that Q is 
nondegenerate on the space {£ = : £ G T} contains the complement of a proper algebraic 
subset. This will imply that the set Q3 is dense. To simplify notations, we only consider 
the case of one linear form. Namely, we show that the set of nonzero linear forms i such 
that Q\£=o is nondegenerate contains the complement of a proper algebraic subset. The cases 
of pairs and triples are handled similarly. For a nonzero linear form i, there exists a basis 
{vj : i = 1, . . . ,d — 1} of the space {i = 0} such that the coordinates of the vectors vj are 
rational functions of the coefficients of £. Define a rational function 

0(£) =^det {Qivlv'^):t,j = l,...,d-1). 

For a linear form i such that is defined, Q\e=o is nondegenerate iff 7^ 0. If = 
on its domain, Q\e=o is nondegenerate only for £ in a proper algebraic subset where is 
undefined. This is a contradiction because Q3 is nonempty and open. Hence, 7^ 0. The 
form Q\e=o is nondegenerate for £ E {£ : (f){£) 7^ 0}. This set is the complement of a proper 
algebraic set. 

To prove that Q4 is dense, we show that the set of nonzero linear forms £ such that 
{Q\i=o, L\e=o) is of type (I) contains the complement of a proper algebraic subset. The form 
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{Q\e=o, L\i=o) is of type (I) iff 

det(Q|,=o) 7^ 0, det{Q\{L=o,e=o}) ^ 0, L\e=o ^ 0. 

Note that the first inequahty and the third inequahty define sets that contain the complements 
of proper algebraic subsets. Hence, it is enough to prove that the set of linear forms i that 
satisfy the second inequality contains the complement of a proper algebraic subset. There 
exists a basis {w^ : i = 1, . . . ,d — 2} of the space {L = 0, £ = 0} such that the coordinates of 
the basis vectors are rational functions of the coefficients of i. Define a rational function 

m =^ det {Qiwl w'^):t,j = l,...,d-2). 

For i such that ip^i) is defined, Q|{j;^=o/=o} is nondegenerate iff ip^i) ^ 0. Therefore, we 
deduce as above that the set of i such that Q|{l=o/=o} is nondegenerate contains the comple- 
ment of a proper algebraic subset. This implies that is dense and proves the lemma. □ 

Denote by tt : M*^ x M'^ x M'^ ^ M'^ x M'^ the projection on the first two coordinates, and by 
p : M'^ X M'^ X M'^ M'^ the projection on the first coordinate. 

Lemma 6. Suppose that (/i, "n^i), (/2, ^2), (mi, 7712) G tt{Q). Then there exists a rational 
linear form n such that 

(/i, mi, n), (^2, m2, n), (mi, m2, n) G ^. 

Proof. As in the previous lemma, one shows that if (/,m) G 7r(r2), the set of n such that 
{l,m,n) satisfies conditions (l)-(4) is open and dense. Thus, the set of linear forms n such 
that the triples (Zi,mi,n), {l2,m2,n), and {mi,m2,n) satisfy conditions (l)-(4) is open and 
dense. The condition that Q\{n=o}n{L=o} is indefinite holds on an open set of n. Hence, such 
a linear form n exists. □ 

For a linear form i, consider a linear map from the space of quadratic forms on to 
the space of quadratic forms on {i = 0} defined by 

FiiQ) = Q\e=o. 
This map is rational if i is rational. The kernel of Fi is 

{xii : i = 1, . . . ,d) . 

Proof of Proposition^ Suppose that for any rational linear form / from p{Q), the quadratic 
form {aQ+(3L'^)\i=o is rational for some a and P with (a, P) 7^ 0. This means that Fi{{Q, L^)) 
has a rational subspace of codimension 1, or, equivalently, for some G M such that 
(a, P) 7^ 0, the space {aQ + pF^, Xil : i = 1, ... ,d) is rational. 

Case 1: For any rational linear forms / and m such that (/,m) G 7r(fi), the space 
{Q, L^, Xil, XjTTi : i,j = 1, ... ,d) is rational. 

We will use the following claim: 

Claim. Let S be not in {Q^F^). Then there are rational linear forms I and m such that 
{l,rn) G 7r(f2), and S is not in 

{Q,L'^,Xil,Xjm : i,j = 1, . . . ,d) . 
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Proof. Suppose not. Then for any rational linear forms I and m such that {l,m) G 7r(i7), we 
have S = a{l, m)Q + m)L'^ on the subspace {Z = 0} fl {m = 0} for some m), (3(1, m) G 
M. First, we show that a(/,m) and (3{l,m) are independent of {l,m) on a large set of pairs. 
Let {1, 171,71) G Q for some rational linear forms /, m, n. Because of the condition (3), Q has 
rank at least 2 on the space 

V =^ {/ = 0} n {m = 0} n = 0}. 

Therefore, Q\v and L'^\v are linearly independent. It follows that 

a{l,m) = a{7n,n) = a{n,l) and [3{l,m) = [3{m,n) = (3{n,l). (1) 

Fix (Zq, mo) G 7r(^7) with rational linear forms and m^. Consider the set 

O = {(/,m) G 7r(r]) : (m, mo) G 7r(n)}. 

This is a nonempty open set in 7r(f2). By Lemma El there is a rational linear form such 
that (Zo, 'TJ'O) '^)) {I, n), (m, mo, 7i) G ^2. Then using ([T]), we obtain 

q;(/o, mo) = «(mo, n) = a(m, n) = a{l, m), 

and similarly, P{lo,77io) = P{l,m). Hence, the coefficients a(/,m) and /3(Z,m) are constant 
for rational linear forms / and m such that (/, m) G (9. Thus, S — aQ — I3L?' = on the space 
{Z = 0} n {m = 0} for rational linear forms / and m such that (/,m) G O. It follows that 
S — aQ — (3L'^ = on an open subset of M'^. Hence, S = aQ+jSL"^. This is a contradiction. □ 

Using the claim, we conclude that the space {Q, L^) is an intersection of spaces of the form 
{Q, L^, Xil, Xjm : i, j = 1, . . .n) for some rational linear forms / and m such that (/, m) G O. 
By the assumption of Case 1, each of these spaces is rational. Hence, the space {Q, L"^) is 
rational too. This gives a contradiction. 

Case 2: For some rational linear forms / and m such that (/,m) G 7r(fi), the space 
(Q, L"^, Xil, XjUi : i,j = 1, ... ,d) is not rational. 

By the assumption, the space {aQ + (3L'^, Xil : i = 1, . . . ,d) is rational for some a, /? G M 
such that {a,P) ^ 0. Then the space 

i^aQ + /5L^, Xil, XjTn : j = 1, . . . , c?) 

is rational too. It follows from the assumption of the Case 2 that the pair {a, (3) with this 
property is uniquely defined up to a scalar multiple. Let Q = aQ + jSL"^. We show that Q 
is proportional to a rational form. The pair {a, (3) can be chosen such that the form Q\i=o is 
rational. 

Because of the uniqueness of {a, (3), Q\k=o is proportional to a rational form for any rational 
linear form k G p{^) H {l,m). Note that the later set is not empty because / and m is in it. 
Let {cj : i = 1, . . . , d — 2} be a rational basis of {/ = 0} ft {m = 0}. We complete it to a 
rational basis of by vector e^-i and such that l{ed-i) = 0. Let 

Ut = {Ci, Cd-i + ted:i = l,...,d-2) for t G Q. 

Since Uq = {I = 0} and / G p{^), a rational form kt that defines Ut is in p{VL) for sufficiently 
small t. Also kt G (/,m) because 

{/ = 0} n {m = 0} C Uf 
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Therefore, atQ\ut is rational for some ctf 7^ when t is rational and sufficiently small. Since 
(/,m) G vr(i7), Q 7^ on the space {/ = 0} fl {m = 0}. Take a rational vector x G {/ = 
OJ n {m = 0} such that ^ 0. Then g(x) G Q, and atQ{x) G Q. Therefore, at G Q, and 
Q\ut is rational for sufficiently small t. In particular, when t = 0, Q{ei,ej) G Q for i,j < (i. 
Also 

Q(ei, ed-i + ted) = Q{ei, Cd-i) + tQ{ei, e^) G Q, 
so that Q{ei, e^) G Q for i < (i. Finally, 

Q(erf_i + terf) = Q{ed-i) + 2tQ{ed-i, e^) + t^Q{ed) G Q. 
Thus, Q{ed) G Q. This shows that Q is rational which is a contradiction. □ 

4. Pairs of type (I) 

In this section we consider a pair {Q,L) of type (I) and dimension 4. By Proposition El 
the pair {Q, L) is equivalent to either 

(2,2 2 2 N (2 2 2 2 \ 

We consider the first case. The other case can be done the same way. Let 

Qq = x\ + x\ — xl — x\ and Lq = x^. (2) 

Let 



H 



S0(2,l) 











CSL(4,C). (3) 



Note that the group H leaves the pair {Qq, Lq) invariant. 
First, we collect some simple facts. 

Proposition 7. Let F be a connected semisimple algebraic subgroup of SL{4:, C) which acts 
irreducibly on such that 

H CF C SL{AX)- 
Then G = SO{S, C) for some nondegenerate quadratic form S. 

Proof. Let f be the Lie algebra of F. For convenience of the reader, let us reproduce in Table 
1 the list of complex semisimple Lie algebras of dimension up to 14 and dimensions of their 
faithful irreducible representations (see \0V\ ) . Note that an irreducible representation of a 
semisimple algebra is a tensor product of irreducible representations of simple factors. One 
can see that f should be of type Ai + Ai or C2. The 4-dimensional irreducible representation 
of the algebra of type C2 is simplectic. On the other hand, H G F acts irreducibly on a 
3-dimensional subspace, and leaves invariant a nonzero symmetric form on this subspace. 
This shows that F cannot leave a nonzero symplectic form invariant. Thus, f is of type 
Ai + Ai. Since f has a unique irreducible 4-dimensional representation, this representation 
is equivalent to the representation of so(4,C) on C^. This means that f = gso{AX)9'^ for 
some g G SL(4, C). Equivalently, f = so{S,C) ^oi some nondegenerate quadratic form S. 
Since F is connected, F = SO(S', C). □ 

Lemma 8. Let G C GL{d,C) be an algebraic group that acts irreducibly on C^. Then its 
unipotent radical is trivial. 
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dimg 


dim p 


Ai 


3 


2,3,4,... 


Ai + Ai 


6 


4,... 


Ai + Ai + Ai 


9 


8,... 


Ai + Ai + Ai + Ai 


12 


16,... 


A2 


8 


3,8,... 


A2 + A1 


11 


6,... 


A2 + Ai + Ai 


14 


12,... 


B2 = C2 


10 


4,5,... 


B2 + A1 


13 


8,... 


G2 


14 


7,... 



Table 1. Complex irreducible representations 



Proof. Let U be the unipotent radical of G, and V = C . Consider the space 

= {x eV -.Ux = x}. 

Since U is unipotent, 7^ 0. Since U is normal in G, is G-invariant. Thus, = V, 
and U = 1. □ 

Lemma 9. Let Q be a quadratic form on which is invariant under the action of H (H 
is defined in Then Q = aQo + PL^ for some a, P E C, where Qo and Lq are defined in 

Proof. Let us write Q = X2, X3, 0) + L{xi,X2,X3)x4 + (3x\ for some linear form L and 

/? G C. Then ■, ■, 0) and L is S0(2, l)-invariant. It follows that ■, ■, 0) = a(5o(', 0) 
for some a G C and L = 0. □ 

The following proposition is the main result of this section and a partial case of Theorem 

HI 

Proposition 10. Let Q he a nondegenerate quadratic form, and L be a linear form such 
that {Q{x), L{x)) = {XQo{gx), fiLo^gx)) for some g G 5'L(4, M) and A, /i G M — {0}, where Qq 
and Lq are defined in Suppose that for every 7^ (0,0), with a, (3 G M, the linear 

combination aQ + /3L^ is not rational. 

Then {{Q{x),L{x)) : x G P(Z^)} is dense m M^. 

Proof. Without loss of generality, A = = 1. Let G = SL(4, C) and if be as in Q . We study 
the action of on G^/Gz- By Ratner's Theorem fRaQlj, the closure H^gGi C Gm./Gz is a 
homogeneous space. Moreover by the result of Shah Sh91, Proposition 3.2], H^gGz = gF^Gi 
where F is the smallest algebraic Q-subgroup containing g~^Hg, and the radical of F is 
unipotent. Here F^ denotes the connected component of Fjj with respect to the Euclidean 
topology. Since H is connected as an algebraic group, F is connected too. 

First, we consider the case when F acts irreducibly on C^. By Lemma |H1 F is semisimple, 
and by Proposition [71 F is one of the subgroups g~^Hg, G, SO(S', C) for some nondegenerate 
quadratic form S. Since F acts irreducibly on C^, F 7^ g'^Hg. Suppose that F = SO(S'). 
Since F is defined over Q, the quadratic form S is proportional to a rational form (see, for 
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example, |Bo95j V By Lemma IHl S = aQ + /3L'^ for some a,P E C. This is a contradiction. 



Thus, F = G. We conclude that H^gGz = G^/Gz- Then 

{iQ{x),L{x)):xeVm} 

= {{Qo{hg-fei),LQ{hg-fei)) : h e iJR,7 G Gz} 

= {(Qo(aei), Lo(aei) : a G Gr} = {{Qo{x), Lo{x) : x G M^} = M^. 

Here ei = (1, 0, . . . , 0), and the last equality holds by Lemma El 

Now we assume that the action of F on is not irreducible. Then the action of F on the 
dual space C is reducible too. The dual action of H has irreducible components {xi, X2, x^) and 
(0:4). Moreover, for any linear form / ^ (3:4), the smallest if-invariant subspace containing / 
should contain (xi, X2, 2:3). Therefore, (xi, X2, X3) and (X4) are the only nontrivial if-invariant 
subspaces. For i = 1, . . . ,4, define linear forms ii{x) = {gx)i. Then 

Q = il + il-il-il and L = £4. 

It follows that £1 (^1,^2,^3) and £2 '=■ (^4) are the only candidates for nontrivial F- 
invariant subspaces. Let F be semisimple. Then its action is completely reducible. Therefore, 
Ci and £2 are F-invariant, and since F has no nontrivial characters, the space of F-fixed 
vectors = £2 = {L). Using that F is a Q-group, we conclude that (L) is a Q-subspace. 
This means that L is proportional to a rational form, which is a contradiction. Therefore, 
F is not semisimple, and its unipotent radical U ^ 1. Note that U is defined over Q. The 
subspace of [/-fixed vectors is F-invariant, so that either £^ = £1 or £^ = £2 = (L). If 
the later holds, L is proportional to a rational form by the same argument as above. Thus, 
we may assume that £^ = £1. In particular, £1 is a Q-subspace. Since F is defined over Q, it 
has a Levi subgroup Fq which is defined over Q. It follows from the description of F- invariant 
subspaces that Fq acts irreducibly on £^. There exists an Fo-invariant complement for £^ 
generated by a linear form Eq. We have that £^° = {io). In particular, (^0) is a Q-subspace, 
and the linear form io can be chosen to rational. By Malcev's theorem, g~^Hg C u^^Fqu for 
some u E U. The forms L{u~^-x) and io{x) are both fixed by Fq, so that L{u~^-x) = a-iQ{x) 
for some a G M — {0}. 

Suppose that g^^Hg = u^^Fqu. Then Fq stabilizes the pair 

[Q{u-^ ■ x), L{u-^ ■ x)) = [ll + a'^il, aio) 

Since Fq is a Q-group, the space of quadratic forms that are fixed by Fq is a Q-space. By 
Lemma El this space is spanned by ^1 + ^2 ~ ^3 ~ '^^^0 ^o- Therefore, for some a, /? G M, 
the form 

is rational. Since the space £1 = (i'1,^2,^3) is rational, there exists a rational vector xq 7^ 
such that £i(xo) = i2{xo) = isixo) = 0. Clearly, 4(xo) 7^ 0. Then Q(xo) = [3 ■ io{xof G Q. 
Therefore, /3 G Q, and the form Q + L"^ = £f -|- ^2 ~ ^3 is proportional to a rational form, 
which is a contradiction. 

Now we may assume that g~^Hg C u~^Fqu. Then 



SL(3) 











gu \ 
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Also 

An element u E U acts on C as follows: u ■ i = i for i E Ci, and m ■ £4 = £4 + for some 
linear form G £1. Then the adjoint action of Fi on U corresponds to the usual action of 
Fi on the space spanned by linear forms iu, u E U. It follows that for every i E Ci, there 
exists u E U such that u ■ £4 = £4 + i. 

Let {ci : 2 = 1, . . . ,4} be the standard basis of Fix (a, 6) E M^. Take x E P(Z^) - 
{g~^e4). By the previous remark, there exists u E such that 

L{ux) = [u ■ (i4){x) = b. 

Write ux = xi + X2 for xi E {ei, 62,63) and X2 E {64). Since Q is indefinite on the 
subspace (ei, 62, 63), there exists f E Fi such that Q{fxi) = a — Q{x2). Then 

Qifux) = QU'xi) + Q{x2) = a. 

Since 



g ^H^gGz — F^Gi — FmU^Gz, 
there exist hn E g^^H^g and 7„ E Gz such that hn'jn ^ /m as n — > 00. Finally, 

{Q{'ynX),L{'ynX)) = {Q{hnlnX),L{hnlnX)) 

(Qifux), L{fux)) = (a, b). 

The proposition is proved. □ 

Remark 11. Proposition fTIll combined with Proposition 0] implies Theorem^ for dimension 
d>5. 

5. Pairs of type (II) 

Now we prove Theorem Q for pairs of type (II) of dimension 4. This will finish the proof 
of Theorem ^ Let 

Qo = X1X4 — X2X3 and Lq = X4. (4) 

Note that we use different (cf. Proposition ^ canonical form to simplify calculations. 
Let 

/ 1 a b ab \ 
10 6 



H 



: a,6 G C S C SL(4,C). (5) 



1a 

\ 1 y 

It is easy to check that Qoijix) = Qo{x) and Lo{hx) = Lo{x) for h E H. 

Lemma 12. The only nontrivial H -invariant subspaces of the dual action of H (i.e the action 
on the space C of linear forms) are 

• (3^4) ■ 

• {ax2 + Pxs, X4) for some a, /5 G C. 

• (X2,X3,X4). 
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Proof. One can check that X4 is the only fixed vector of H (up to a scalar multiple). Since 
the action of H is unipotent, every nontrivial if-invariant subspace V contains a nonzero 
vector fixed by H. It follows that {X4) C V. Consider a factor space £/ (X4). The subspace 
{x2,X3,X4) I {xi) consists of if-fixed vectors, and for any v ^ (x2,X3,X4), one has {Hv,X4) = 
C The conclusion follows. □ 

Lemma 13. 1. Let G be a simple group, and G = G x G. Denote by tti and tt2 the 
projection maps. Let H < G be such that T^i{H) = G for i = 1,2. Then H = 
{{a{g),g) : g G G} for some automorphism a of G. 
2. Let g = sl(2,C), and g = 5 © 5. Denote by tti and 112 the projection maps. Let 
i) < Q be such that 7ii{l)) = Q for i = 1,2. Then I) = {{Ad{g)x,x) : x G g} for some 
geSL{2,C)- 

Proof. Let 

Si = {geG: ig,e) G H} and S2 = {g E G : (eg) G H}. 
It is easy to check that Si and ^2 are normal subgroups of G, that is, g'^Sig C Si for every 
g E G, i = 1,2. Since G is simple. Si is either {e} or G. 

Suppose that Si = G. We show that H = G. Let {g, h) G G. For some gi G G, {gi, h) G H . 
Then 

{9,h) = {gi,h){g^^g,e) E H. 

Hence, H = G. Similarly, H = G ii S2 = G . 

Suppose that Si = S2 = {e}. For any g E G, there exists a unique element a{g) such that 
{a{g),g) E H. Because of the uniqueness, a is a homomorphism. It is surjective because 
7fi{H) = G and injective because S2 = {e}. This proves the first part of the lemma. 

It is straightforward to rewrite this argument for simple Lie algebras. It is known that any 
automorphism of s[(2, C) is inner. Thus, the second part of the lemma follows. □ 

The following proposition finishes the proof of Theorem ^ Its proof is similar to the proof 
of Proposition ITUl 

Proposition 14. Let Q be a nondegenerate quadratic form, and L be a linear form such 
that {Q{x), L{x)) = {\Qo{gx), fj,LQ{gx)) for some g E SLi^A, M) and A, /i G M — {0}, where Qo 
and Lq are defined in Suppose that for every {a, (3) 7^ (0,0), with G M, the linear 
combination aQ + f3L'^ is not rational. 

Then {{Q{x),L{x)) : x E V{Z^)} is dense m M^. 

Proof. We may assume that A = /i = 1. Let G = SL(4, C), and be as in (0). Consider the 
action of on G^/G^. As in the proof of Proposition [TU[ we know that H^gGz = gF^Gz 
where F is the smallest algebraic Q-subgroup containing g~^Hg. The group F is connected as 
an algebraic group, and its radical is unipotent. There is Levi decomposition F = FqU where 
Fq is a connected (as an algebraic group) semisimple Q-subgroup, and U is the unipotent 
radical of F. Note that U is defined over Q. 

To prove the proposition, it is enough to show that 

{{Qifz), L{fz)) ■.fEFl,zE V{Z^)} = R'. (6) 

Consider the action of F on the space of linear forms C. Let V be a nontrivial F-invariant 
subspace. Since H is unipotent, V contains a nonzero vector fixed by g^^Hg. By Lemma 
IT^ the only vector fixed by g~^Hg is L (up to a scalar multiple). Thus, (L) C V. It follows 
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that there is a unique F- irreducible subspace V C £. Namely, it is the intersection of all 
nontrivial F-invariant subspaces. This subspace is contained in every F-invariant subspace. 
Also V is defined over Q. Indeed, for any a E Gal(C/Q), V" is F(Q)-invariant, and F(Q) 
is Zariski dense in F because F is connected. Therefore, is F-invariant, and V C V^. 
Comparing dimensions, we conclude that V = V^. This shows that V is defined over Q. 

Let be the subspace of vectors fixed by U. Since U is normal in F, the space is 
F-invariant, so that V C . It follows that Fq act irreducibly on V. Since Fq is semisimple, 
the F-action on is completely reducible. Suppose that V C . Then = V © W 
for some F-invariant subspace W. However, this contradicts the description of i7-invariant 
subspaces in Lemma IT^ Thus, V = . 

We can write 

for some real (Fo)iR-invariant subspace Wr. Then C = V Q) W where W = Wr (S) C is 
Fo-invariant, because (Fo)k is Zariski dense in Fq. Note that W is defined over M. Let 

V = {veC^: i{v) =0 for £ e V} 

and 

W = {veC^: i{v) = for £ G W}. 

Clearly, = V"©!^, V is F-invariant, and W is Fg-invariant. Moreover, W is Fg-irreducible 
because W ~ C^/V as Fo-modules, and any nontrivial Fo-invariant subspace of C^/V would 
give by duality a nontrivial Fg-invariant subspace in V. The space (C'^/V)'^ is nonzero and 
Fo-invariant. It follows that (C^/V)^ = C^/V, so that U acts trivially on C^/V. 

Let ii{x) = {gx)i for z = 1, . . . , 4. Then Q = iii^ — £2^3 and L = £4. 

Consider several cases: 

Case 1: dimV = 1. Then V = (L), and since V is defined over Q, L is a multiple of a 
rational form. This is a contradiction. 

Case 2: dimV = 2. Then dim V = dim W = 2. Since Fo is semisimple, the action of Fo 
on V is either trivial or irreducible. 

Denote 

A = {g e SL(4, C) : g\v = id, g\w = id + a for a G End(W, V)}. 

Suppose that Fq acts irreducibly on V. Then U acts trivially on V because otherwise 
7^ is a nontrivial Fo-invariant subspace. Since it was shown above that U acts trivially 
on C^/V too, this implies that U C A. The Lie algebra fo C sl{V) x s[(Vr) of Fo satisfies the 
conditions of Lemma IT^ unless fo = sl{V) x sl(W). 

First, we consider the case when the last equality holds. Then A is an irreducible Fo- 
module. Suppose that U = 1. Then V and W are F-invariant subspaces such that V + W = 
C Since every two 2-dimensional subspaces in Lemma ^1 are contained in the unique 3- 
dimensional subspace, this gives a contradiction. Thus, [/ 7^ 1, and since it is a submodule 
of the irreducible Fo-module A, we conclude that U = A. In particular, for any x ^ Vr and 
any v G Vr, there exists m G [/r such that ux = x + v. We will use this fact latter. 

Now we assume that fo is a proper subalgebra of sl(y) x 5l(W). Then by Lemma IT^ 

fo = {{(f)-^x(j),x) -.xesliW)} 

for some isomorphism (p : V W. Since fo is an M-subalgebra, can be taken to be an 
M-map. Let a ^ {a G End(lV, V")} be the Lie algebra of A. The adjoint action of fo on a is 
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isomorphic to adjoint action of s[(iy) on gl(iy). The isomorphism is a ^ o a. It follows 
that a has the only nontrivial fo-modules: {A</)~^ : A G C} and {(f)~^ o b : b E si{W)}. 

Let U correspond to {A^""*^ : A G C}. Take a basis {vi, V2} of V. Then {vi, V2, 0(^2)} 
is a basis of C^, and with respect to this basis, 



9 ^9 
9 



: g G SL(2),A G C 



Let S be an F-invariant quadratic form. Then the subspace V and W are totally isotropic 
with respect to S. Thus, the matrix of S is of the form ^ ^ ^ ^ some matrix X such 

that ^gXg = X for all g G SL(2). It follows that X = ( ^ rT ) ^'^^ some m G C, and 



the quadratic form S is unique up to a scalar multiple. Since F is a Q-group, the space of 
F-invariant quadratic forms is defined over Q. Therefore, is a multiple of a rational form, 
and g~^Hg is contained in the Q-group SO(S'). Now we can argue as in the Case 4 when F is 
of type Ai + Ai (see below) to get a contradiction. Hence, we may assume that U corresponds 
to {0^^ ob : b E sl(W)}. Then for any x ^ Vk and any v G V]r, there exists u E such that 
ux = X + V. We will use this fact latter on. 

Suppose that Fq acts trivially on V. If U acts trivially on V, F has linearly independent 
F-invariant vectors. They correspond to distinct 3-dimensional F-invariant subspaces in C 
This contradicts Lemma IT^ Thus, U acts nontrivially on V. Let f 1 G be a [/-fixed vector 
in V. Take V2 such that V = {vi,V2)- Let 

B = {g e SL(4, C) : gvi = Vi,gv2 = V2 + tvi, g\w = id for t G C}. 

Clearly, U C AB and U ^ A. U U ^ B, then the subspaces V and W would be F-invariant, 
which contradicts Lemma fT^ Thus, U <^ B. Let u = ab E U for a G A — {1} and b G B — {1}. 
Note that A is abelian, and F normalizes A. Thus, 



u 



{u n A)u = b-\u nA)bcunA. 



Since the action of B is algebraic, it follows that B normalizes U. Let 



a 



1 a 
1 



for some d G End(PV, V) — {0} with respect to the decomposition V © W. Since Fq acts 
irreducibly on W, the restriction map is Fq SL(W) is surjective. For g G SL(l^), take 
/ G Fq such that f = id® g with respect to the decomposition V © W . Then 



Thus, 

1 dg — a 
1 



G U 



for every G SL(W^). In particular, it follows that U r\ A 7^ 1. The only nontrivial BFq- 
submodule of A is 



A 







End(iy, (wi)) 
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Suppose that U C AqB. Then the hnear form corresponding to the projection to {V2) is fixed 
by F. This contradicts Lemma [T21 Hence, there exists aibi G U for ai E A — Aq and bi G B. 
Write 

1 Si 
1 



ai = 

for some di G End(W^, V) - End{W, (fi)). As above, 

G f/ 



1 aiQ — Si 
1 

for all g G SL(Vr). One can choose g such that dig — di ^ End(W, (fi)). This shows that 
U n A = A. It follows that for any x ^ Vs. and any v G V^, there exists u G [/r such that 

UX = X + V. 

Finally, we finish the proof in the Case 2. By Lemma IT^ 
for some a, /5 G C. Let f 7^ be a vector such that 

e,{v) = £3(1;) = i4{v) = 0. 

Then v E V H V-^, and since Q is nondegenerate, it follows that W <^ {v) . Note that 
L\v = iilv = 0. In particular, L\w ^ 0. Fix (a, 6) G M^. Take z G P(Z^)-\/. Let z = z' + z" 
for G y and z" G IV - {0}. Since L\w 7^ 0, there exists g G SL(W^ir) such that L{gz") = b. 
Also we can choose gn G SL(IV]r) such that gn 9 and Q{gnz",v) 7^ 0. Take /„ G Fgjg such 
that fn\wR = S'n and m„ G [/k such that 

^here = gfj^. Then 

Q{UnfnZ) = QifnZ + t„t^) = Ql/n^^) + 2tnQ{fnZ, v) 
= QifnZ) + 2tnQ{gnZ", v) = tt. 

Also 

L{UnfnZ) = L{fnZ' + fnZ" + t„f ) = L{gnZ") b. 

This shows (0). 

Case 3: dimV = 3. Let V = (v). By Lemmad V = (4, ^3, h)- It follows that g(v) = 0. 
Since F has no nontrivial characters, Fv = v. We show that is a Q-subspace. For any 
a G Gal(C,Q), W is Fo-invariant (because Fq is connected). Since W fl W ^ {0} and W 
is Fo-irreducible, it follows that W = W^. Thus, W is defined over Q. 

For any z G — {0}, define 

F,^ = {geFS^:Qigz,v)^0}. 

We claim that Fq is not empty. Suppose that Q{gz,v) = for every g G Fq^. Since Fgjg is 
Zariski dense in Fq, {Fq^z) = W^.. Thus, C (f )"*". Since Q{v) = 0, the vector v lies in the 
radical of Q. This gives a contradiction. Thus, Fq is a nonempty Zariski open subset of Fq^. 
It follows that Fq is dense in Fqjj in Euclidean topology. Put 

F,-' = f]{F^:zeV{W^)}. 
This set is dense in Fq-^ by Baire Category Theorem. 
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Consider the orbit O = [L]^or in the real projective space P(PF]^) where is the dual 
space of W]^. If O consists of rational points, then [L]^or = [L] and [L]^° = [L], which is a 
contradiction. Thus, there exists g G -Fqjj such that the linear form L{gx), x G W , is not 
proportional to a rational form. Then L{gV{Wi)) is dense in M. Therefore, L{Fq^V {Wj)) = 



Every element u E U acts on as follows: uv = v and for w G W, uw = w + lu{w)v 
for some linear form /„ on W . Using this notations, the action of Fq on U by conjugation 
corresponds to the usual action on the space of linear forms spanned by /„, u G f/. Thus, 
this action is irreducible. Note that U ^ 1 because = V. It follows that for every linear 
form / on W, there exists u E U such that uw = w + l{w)v for w G W. In particular, for any 
w eW and t G M, there exists u E Uu such that uw = w + tv. 

Now we are ready to finish the proof of Case 3. Fix (a, b) G M^. There exist (?„ G and 
Zn G V{Wi) such that L{gnZn) b. Take Un G Um. such that Ungn^n = dnZn + tnV where 



This shows (jHl). 

Case 4: dimV = 4. By Lemma |H1 F is semisimple. From Table 1, F is one of the types 
Ai, Ai + Ai, C2- The first case is impossible because F contains a 2-dimensional unipotent 
subgroup. Denote by f the Lie algebra of F. 

Let F be of type C2. Then f = gsp{4:,C)g~^ for some g G SL(4, C). Equivalently, f = 
sp{S, C) for a nondegenerate symplectic form S over C. The form S is proportional to a real 
form because F is defined over M. Fix (a, 6) G M^. By Lemma 121 there exists x G M'* such 
that Q{x) = a and L{x) = b. Take x„ G — {0} such that x„ — > x, and ^ — (xn) 
such that S\(^xn,yn) is nondegenerate. Let Vn = {xn,yn)- Take Zn G P(Kiz)- Every element of 
the form g Q) id for g G SL(14r) with respect to decomposition Vn © is in Sp(S', M) for any 
5f G SL(V^k). Thus, there exists /„ G Fj^ = Sp(S', M) such that x„ = fnZn- Then Q{fnZn) — ^ a 
and L{fnZn) — > This shows 

Let F be of type Ai + Ai. Using an argument as in Proposition [7J F = SO{S) for some 
real nondegenerate quadratic form S. 

Let f) be the Lie algebra of H. It is easy to check that the normalizer of f) in s[(4, C) is 



M and L{F;^V{Wz)) = M. 



Then 



QiUngnZn) = Q{gnZn + tnV) = Q{gnZn) + 2tnQ{gnZn, v) = O, 



and 



LiUngnZn) = HgnZn) b. 




lux y 



-t; 

\ 



X 



y 



t \ 



u ) 




(7) 



and the centralizer of f) in sl(4, C) is 




I ^ X y t\ 

y 

X 

\ y 




(8) 
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Since contains closed unipotent subgroup g^^H^g, it cannot be compact. Thus, is 
isomorphic to either S0(3, 1)m or SO(2,2)]r. Recall that S0(3, 1)m is isogenous to SL(2,C), 
and SO(2,2)i8 is isogenous to SL(2,]R) x SL(2,]R). In both cases, the group g~^H^g is a 
maximal unipotent subgroup of F. Let t)K be the Lie algebra of H^. Consider the map 

: NsLii,M.){gH^g''r ^ GL(Ad((7)f)M) 

defined by (j){n)h = Ad{n)h. By ([7j) and (jSI), the image of (p is isogenous to M>o x ]R>o- If -^m 
were isogenous to SL(2,C), then (f){NFj^{gH^g~^))° would be isogenous to ~ S0(2,]R) x 
M>o. To check the last statement, one can note that it is obvious when g~^H^g is the subgroup 
of upper unipotent matrices in SL(2,C), and g~^H^g is conjugate to this subgroup. This 
shows that S has signature (2, 2). 

Denote E = SO(Qo) and G = SL(4, C). Then F = g~^g^^Egig for some gi G G. First, we 
show that gi E ECg{H). We have H and giHg^^ C E. Since maximal unipotent subgroups 
are conjugate, eHe~^ = giHg^^ for some e E E. Thus, e~^gi G Ng{H), and gi G ENg{H). 
Without loss of generality, gi G Ng{H). Consider the map ip '■ Ng{H) GL([)) defined by 
tp{n)h = Ad{n)h. Since this map is algebraic, its image is an algebraic subgroup in GL(f)). 
By ((Tj) and (jH)), dim iP{Ng{H)) = 2 and iP{Ng{H))° is generated by diagonal matrices. Thus, 
i){NG{H))° is abelian. Let 

T = {diag(M, V, f u^'^) : u,v E C^}. 

This is a maximal torus of E. Since ip{T) has dimension 2, it is a maximal torus of GL(f)), 
and so is ip{gi^Tgi). Since they commute, 

i.e. ip{gi) normalizes ip{T). The normalizer of V'(T) in GL(()) is generated by ip{T) and 
the transformation that permutes two elements of the basis of f). It is easy to see that this 
transformation is in ip{E). Thus, ip{gi) E ip{E). Since = Cg{H), it follows that 

g, E ECg{H). 

It follows from Lemma IT^ that H has a unique fixed vector v (up to a scalar multiple). 
Then for c G Cg{H), cv = Xv for some A G C. Suppose that /i G C — {A} be an eigenvalue of c. 
The complex eigenspace corresponding to fi and A are if- invariant. Each of these subspaces 
contains a nonzero vector fixed by H (because H is unipotent). This contradicts Lemma 
[121 Thus, c has a unique eigenvalue, and c = (A/)co where / is the identity matrix, and 
Co G Cg{H) is unipotent. Let Cq be the set of unipotent elements of Cg{H). If Ci,C2 G Cq, 
civ = C2V = V, and Cic^^f = v. Then Cic^^ is unipotent. Thus, Cq is a subgroup. We have 
Cg{H) = Z{G)Co. Clearly, Cq is unipotent algebraic subgroup, so that it is connected. By 
dHl), Co = Hue where 

/ 1 t \ 
10 

^* ~ 10 ■ 
\ 1 y 

Thus, gi E EZ{G)u(c- Then 

F = g-'g^'Eg,g = g-\-'^0{Qo)utg = g-'SO{Qo + txl)g 

= so(g + tL2) 
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for some t G C. Since F is a Q-group, the quadratic form Q + tL"^ is proportional to a rational 
form. This is a contradiction. The proposition is proved. □ 

6. Conclusion 

Let [Q, L) be a pair such that Q is a nondegenerate quadratic form, and L is a nonzero 
linear form. It would be interesting to obtain neccesary and sufficient conditions for the 
property 

{{Q{x),L{x)) : X eViZ'')} = R^ 
to hold. In this context, we formulate the following conjecture: 

Conjecture 15. Let Q be a nondegenerate quadratic form, and L a nonzero linear form in 
dimension d > A. Suppose that 

1. For every /3 G M, Q + jSL"^ is indefinite. 

2. For every {a,j3) ^ (0,0), with G M, aQ + /5L^ is not rational. 
Then {{Q{x),L{x)) : x G P(Z^)} is dense in M^. 

The first condition in the conjecture is neccesary for the density to hold. Indeed, suppose 
that Q + /?L^ is definite (say, positive definite) for some /3 G M. By Proposition |21 

Q = £l + ... + el_, - f3£l and L = (9) 

for some linearly independent linear forms £{, i = 1, . . . ,d. If G is an accumulation 
point of the set {{Q{x), L{x)) : x G Z^}, then G M"' is an accumulation point of the set 
{{ii{x), . . . ,id{x)) : X G Z'^} which is impossible. 

Due to Theorem n it remains to prove Conjecture ^1 in the case when {Q,L) is of type 
(II), and Q\l=o is positive definite, i.e. 

Q = il + ...+ + and L = id (10) 

for some linearly independent linear forms ii, i = 1, . . . ,d. The method of the proof of Propo- 
sition with minor modifications allows to prove Conjecture in dimension 4. However, 
one should keep in mind that the method of reduction to lower dimension (Proposition ^ 
fails to work in this case. Namely, for every d > 4, there exist pairs {Q, L) of dimension d 
that satisfy the conditions of Conjecture UHl but for every rational subspace V of codimension 
1, the set {(Q(x), L{x)) : x G Vz} is not dense in M^. To construct such an example, we take 
a pair {Q,L) as in ^TU^ such that the space {ii, . . . , id-2, ^d) contains no nonzero rational 
forms, and {Q, L) satisfies the second condition of Conjecture [131 Clearly, such pairs exist. 
In fact, such pairs are generic in suitable sense. Let £ be nonzero rational form. It is easy 
to see from Lemma IHthat the pair {Q\(,=q, L\i=q) is of type (II) iff Q\w is degenerate where 
W is the space {£ = 0, L = 0}. The forms i = I, ... ,d — 2, are linearly independent 
because i ^ . . . , id-2, ^d)- Therefore, 

is nondegenerate, and (Q|^=o, -^^|£=o) is of type (I). Since Q\l=o is positive definite, the pair 
{Q\i=Oy L\e=o) is as in 0. As we saw above, density fails in this case. 
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